In many circumstances, it is desirable to process signals in digital form if possible, because operations on digital data are essentially perfectly reproducible from one circuit to another whereas, by contrast, analog circuits are corrupted by a variety of practical imperfections (e.g.—offsets, thermal noise, signal coupling, distortion, etc.). Accordingly, analog circuits are practical for simple functions, but digital circuits are preferred where complexity is high.
Digital representations of analog signals are sampled at discrete times, whereas analog signals vary continuously. The act of sampling a signal, in itself, delays it; converting it between analog and digital forms involves a series of steps, each delaying the signal; and processing signals comprises a sequence of computations, each of which delays the signal further. A typical digital signal processing system might delay the signal, for example, by ten samples from input to output.
The Nyquist-Shannon sampling theorem shows that the minimum sampling rate required for any sampled system relates to its bandwidth—in a simple case sampling should be at least twice as fast as the bandwidth of the signal. CD audio, for example, is sampled at forty-four point one Hz in order to reproduce twenty kHz of signal, with approximately ten percent of “over-sampling” to make smoothing filters practical. Ten samples of delay at this rate would be over two hundred microseconds.
A known technique for reducing the computational load or power consumption of a filtering system is to decimate the input signal, filter at the resulting reduced rate, and then interpolate the filtered signal back to the original rate. Decimation is usually mathematically described as lowpass filtering (to remove high-frequency energy in the input signal) followed by sub-sampling, but is often implemented with a polyphase filter. A particularly efficient special case is the sinck filter, as described in chapter 13 of “Delta-Sigma Data Converters: Theory, Design and Simulation” by Norsworthy, Schreier and Temes, (IEEE Press) and other references.
Interpolation is similarly mathematically described as the insertion of zero-value samples to increase the data rate followed by a filtering operation to smooth the output. There are several known efficient techniques and, as with decimation, sinc filtering is particularly efficient. As will be apparent, these decimation and interpolation processes delay the signal.
In the area of linear control theory a “plant”, which can be any given physical system, is controlled with a feedback system by measuring the output of the plant, comparing that to a desired output, and feeding the difference back to the input of the plant through some filtering function designed so that the plant output will track the desired output.
A fundamental limitation of these feedback controller systems is the well known “Bode integral”: a classical theorem in linear control theory that states that feedback can only reduce interference at some frequencies by worsening it at others. The improvement, in deciBels, is at best zero when integrated over frequency. The practical situation is even worse when the “plant” is non-minimum phase, e.g. because it contains significant delay.
For example, in “Control System Design, Lecture Notes”, K. J. Astrom, (http://www.cds.caltech.edu/˜murray/courses/cds101/fa02/caltech/astrom.html), Astrom gives a relationship ωgcTd0.4 between delay time Td and the loop-gain crossover frequency ωgc above which control is substantially ineffective. A controller implemented with a technology that produces delay, e.g. from latency in converting between analog and digital forms, will inherently suffer in performance compared to a similar quality controller with less delay.
Continuing the above-mentioned numerical example for CD audio, a controller that has ten microseconds of delay would have ωgc40 krad/s and be substantially uncontrolled above six kHz. The hundreds of microseconds implied for a CD-rate system above would be completely unacceptable.
Latency in a digital controller system can also be a problem for systems that do not use feedback. For example, noise-canceling headsets using adaptive transversal digital filters are described by Widrow in the article “Adaptive Noise Canceling Principles and Applications” (IEEE Proceedings, vol. 63 no. 12, December 1975).
These are feed forward control systems: they sample the interference with microphones outside the headset, estimate the impulse response (or, equivalently, the frequency response) from this interference to a sensor microphone between the loudspeaker and the ear; and synthesize a canceling signal which is then applied to the loudspeaker. In the practical case there is a frequency-dependent response from the loudspeaker to the sensor microphone, which will limit the accuracy of cancellation unless compensated. Latency in implementing a canceller can mean that the “time of flight” of the interfering sound wave from the external sensor to the loudspeaker is less than the latency of the control system: thus interference can pass the loudspeaker before the canceller can react.
U.S. Pat. No. 4,455,675 to Bose et al. discloses the use of an active feedback controller to reduce acoustic noise leaking into a headphone, where it is well known in control theory that interference appearing at the feedback point will be reduced by a factor
      1          (              1        -                  L          ⁡                      (            s            )                              )        ,where loop gain L(s) is the product H(s)C(s) of a term H(s) due to the “plant” (which models physical resonances and delays in the loudspeaker, microphone and in the acoustic and electronic signal paths) and a term C(s) due to the controller. The controller is typically implemented as an analog circuit. Feedback is a robust method of noise reduction in that, unlike with feed forward implementations, there is little sensitivity to the gains of microphones or the loudspeaker.
Controller design for active noise cancellation using pure feedback is difficult in the frequency range at which the human ear canal resonates because the transfer function changes rapidly and unpredictably with frequency in this range. In practical systems, this is mitigated by mechanical absorption of these frequencies in heavy earpieces or by marketing the problem as a feature. The principal advantage of these systems over digital cancellers, such as described by Widrow (discussed above), was that the analog filters are compatible with low-power operation.
As a further example, public address systems, megaphones, feed forward noise cancellers and hearing aids all have performance limited by an undesired acoustic feedback path from their loudspeaker outputs back to their microphones. This path, combined with the intended “forward gain” path from their microphones to their loudspeakers, creates a feedback loop that distorts the frequency response of the system and can even make it oscillate (“howl”). A good way to control this problem is to add an electronic feedback path that models and cancels the acoustic feedback path. However, if it is desired to implement the model and cancellation digitally, the problem of latency in the digital filters again arises.
Similar problems arise in other applications, for example in on-channel radio repeaters where a radio signal is received at one antenna and retransmitted at another at the same frequency, where reflections from objects in the vicinity of the repeater can cause undesired feedback paths with very short “times of flight”.
Digital feedback controllers for mechanical systems are also well known: for example most modern jet aircraft use digital control to manage their flight surfaces. As mechanical systems become smaller, their natural frequency responses become faster and latency becomes more difficult to manage—and simultaneously power constraints become tighter, making “brute force over-sampling” less practical. Read/write heads in disk drives, for example, are mechanically positioned and a feedback controller is used to maintain their centering on a desired track. The natural frequencies of these systems are on the order of kilohertz, making controller latencies on the order of tens of microseconds troublesome.
The emergence of micro-electromechanical systems (of which accelerometers are perhaps the best-known present example) moves the resonances of physical systems up into the megahertz range, so that controller latencies must be kept in the nanoseconds.
Johns and Lewis (“IIR Filtering on Sigma-Delta Modulated Signals”, Electronics Letters v. 27 no. 4 pp. 307-308, Feb. 14, 1991) teach directly filtering delta-sigma modulated signals at an over-sampled rate to minimize latency and hardware complexity. Since latency is well known to be a key difficulty for feedback implementations of active noise reduction, it is apparent that the technique can be profitably applied.
The Johns and Lewis idea can be seen as having two components: over-sampling and delta-sigma modulation. Latency is reduced by over-sampling for the simple reason that data converters delay signals by some number of samples, so that faster sampling naturally reduces latency. Delta-sigma modulation is a technique of representing over-sampled signals so that errors from quantization to a finite number of bits are concentrated at frequencies outside the band of interest. This mitigates the increase in hardware complexity that would otherwise result from over-sampling.
A typical system using this technique might increase sampling rate by a factor of sixty-four to something on the order of three MHz, but reduce the number of bits processed from sixteen to one. The net effect is a complexity increase of approximately a factor of four in exchange for a latency reduction by a factor of sixty-four.
When the physical system being modeled contains a pure transport (“time of flight”) delay, such as that for sound propagation from one acoustical component to another in the headphone example above, over-sampled systems allow for efficient modeling, however Nyquist-sampled systems need several taps to perform interpolation when the physical system has a delay that is not an integer number of samples.
The technical and academic literature teaches filtering in the delta-sigma domain for the case of fixed-coefficient filters. It is known that adaptive filtering is difficult with delta-sigma signals because the nonlinear operations (such as multiplying signals) required by adaptive algorithms mix out-of-band quantization noise energy into the signal band. Feed forward systems are usually based on transversal filters, which require adjustment of one “tap” coefficient for every sample. Increasing the sampling rate increases the number of taps, and also rapidly reduces the rate at which an adaptive system converges (because there are more adjustments to make, and they also interact more strongly). Thus over-sampling is usually contraindicated in adaptive systems, and (as per the previous paragraph) particularly so for delta-sigma over-sampled systems. Over-sampling may also be impractical for wideband systems, where sampling rates are already high.
Delta-domain filtering may also be impractical in cases where cascades of sections having a high-pass or notch character are required, because in these cases the high-frequency noise generated by each section propagates through all following high-pass or notch sections, thus limiting the gain available for the system and wasting dynamic range.
Gao (“Adaptive Linear and Nonlinear Filters”, Ph.D. thesis, University of Toronto November 1991) showed mathematically how to adapt coefficients of recursive (infinite impulse response or “IIR”, in casual usage) filters, and gave an example of how these principles could apply to feed forward linearization of loudspeakers. It is known to be difficult to adapt IIR filters for two key reasons: the search space may have local minima; and there is a danger of instability during adaptation. On the other hand adaptive IIR filters model resonances efficiently, whereas transversal filters are more efficient at modeling transport delays and short impulse responses. Gao also teaches the use of adaptive IIR modeling of loudspeakers to improve their linearity.
Gao's technique of adapting IIR filters uses “backpropagation” of desired signals through filter blocks with a response inverse to the forward path, and allows cascade structures to be adapted. However, the backpropagation technique frequency-weights the signal minimized, which in the presence of interferers can cause misadjustment.
Johns (“Adaptive Analog and Digital State-Space IIR Filters”, Ph.D. dissertation, University of Toronto 1989) also teaches adaptation of IIR filters. He shows a general method which is computationally intensive but applies to a very general class of IIR filter designs, and simpler “single-column” and “single-row” variants that apply in special cases.
Johns also teaches use of analog and digital simulations of singly-terminated LC ladder filters to obtain desirable dynamic-range properties in fixed filter designs (for the special case of spectrally white inputs), and shows that basing single-row and single-column adaptive filters on these single-terminated structures gives good performance in the case where a good initial approximation is available to the desired IIR filter transfer function.
Accordingly, it is known that both feedback and feed forward techniques are sensitive to processing latency, which has made digital implementation of controllers difficult; but analog implementations of controllers are severely limited in the complexity that they can handle and hence their performance is limited and they also have high manufacturing costs.
Delta-sigma techniques promise reduced latency, which helps to make digital implementation of feedback control systems practicable, but may worsen adaptation and the complexity of feed forward control systems.
It is desired to find a method of reducing the latency of digital signal processing for filters without being forced to the high over-sampling ratios and computational penalties of delta-sigma processing, and in which adaptive filtering is not compromised by out-of-band noise.